REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  NO.  0704-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments 
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ORIGINAL  ARTICLE 


Pixel  super-resolution  using  wavelength  scanning 


Wei  Luo1,2,3,  Yibo  Zhang1,2,3,  Alborz  Feizi1,2,3,  Zoltan  Gorocs1,2,3  and  Aydogan  Ozcan1, 2,3,4 


Undersampling  and  pixelation  affect  a  number  of  imaging  systems,  limiting  the  resolution  of  the  acquired  images,  which 
becomes  particularly  significant  for  wide-field  microscopy  applications.  Various  super-resolution  techniques  have  been  imple¬ 
mented  to  mitigate  this  resolution  loss  by  utilizing  sub-pixel  displacements  in  the  imaging  system,  achieved,  for  example,  by 
shifting  the  illumination  source,  the  sensor  array  and/or  the  sample,  followed  by  digital  synthesis  of  a  smaller  effective  pixel  by 
merging  these  sub-pixel-shifted  low-resolution  images.  Herein,  we  introduce  a  new  pixel  super-resolution  method  that  is  based 
on  wavelength  scanning  and  demonstrate  that  as  an  alternative  to  physical  shifting/displacements,  wavelength  diversity  can  be 
used  to  boost  the  resolution  of  a  wide-field  imaging  system  and  significantly  increase  its  space-bandwidth  product.  We  confirmed 
the  effectiveness  of  this  new  technique  by  improving  the  resolution  of  lens-free  as  well  as  lens-based  microscopy  systems  and 
developed  an  iterative  algorithm  to  generate  high-resolution  reconstructions  of  a  specimen  using  undersampled  diffraction  pat¬ 
terns  recorded  at  a  few  wavelengths  covering  a  narrow  spectrum  (10-30  nm).  When  combined  with  a  synthetic-aperture-based 
diffraction  imaging  technique,  this  wavelength-scanning  super-resolution  approach  can  achieve  a  half-pitch  resolution  of 
250  nm,  corresponding  to  a  numerical  aperture  of  ~1.0,  across  a  large  field  of  view  (>20  mm2).  We  also  demonstrated  the 
effectiveness  of  this  approach  by  imaging  various  biological  samples,  including  blood  and  Papanicolaou  smears.  Compared  with 
displacement-based  super-resolution  techniques,  wavelength  scanning  brings  uniform  resolution  improvement  in  all  directions 
across  a  sensor  array  and  requires  significantly  fewer  measurements.  This  technique  would  broadly  benefit  wide-field  imaging 
applications  that  demand  larger  space-bandwidth  products. 

Light:  Science  &  Applications  (2016)  5,  el6060;  doi:10.1038/lsa.2016.60;  published  online  8  April  2016 
Keywords:  holographic  imaging;  on-chip  microscopy;  pixel  super-resolution;  wavelength  scanning;  wide-field  imaging 


INTRODUCTION 

High-resolution  imaging  across  a  wide  field  of  view  (FOV)  is  essential 
for  various  applications  in  different  fields  and  requires  imaging 
systems  to  have  large  space-bandwidth  products.  Ever  since  the 
widespread  adoption  of  CCDs  (charge- coupled  devices)  and  comple¬ 
mentary  metal-oxide  semiconductor  (CMOS) -based  image  sensors  to 
capture  digital  images,  a  tremendous  amount  of  research  and 
development  effort  has  been  devoted  to  optics,  semiconductor 
technologies  and  signal  processing  to  create  high-resolution  and 
wide-field  imaging  and  microscopy  systems.  In  conventional  lens- 
based  optical  imaging  designs,  a  large  space-bandwidth  product  can  be 
achieved  by  using  higher  magnification  and  larger  lenses,  and  the 
image  sensors  are  accordingly  made  larger  in  area  with  higher  pixel 
counts.  Another  approach  is  to  make  image  sensors  with  a  smaller 
pixel  pitch  while  still  maintaining  a  relatively  large  active  area. 
However,  both  of  these  approaches  have  drawbacks:  Larger  optical 
components  make  the  imaging  system  bulky  and  significantly  more 
expensive;  in  contrast,  physically  reducing  the  pixel  size  sacrifices 
the  signal-to-noise  ratio  because  a  smaller  light  sensing  area  is  made 


available  for  each  pixel,  reducing  the  external  quantum  efficiency  of 
the  imager  chip1. 

As  an  alternative,  the  optical  signal  processing  community  has 
provided  a  powerful  framework,  termed  pixel  super-resolution2-6,  to 
obtain  a  high-resolution  image  from  a  series  of  low- resolution  (that  is, 
undersampled)  images.  Pixel  super- resolution  was  originally  developed 
in  lens-based,  point-to-point  projection  imaging  systems2-6  and  was 
later  applied  to  lens-free  and  holographic  imaging  techniques7-12  to 
significantly  enhance  the  space-bandwidth  product  of  the  recon¬ 
structed  images  using  both  CCD  and  CMOS  imager  chips.  In  either 
implementation,  lens-based  or  lens-free,  this  super-resolution  frame¬ 
work  requires  low-resolution  undersampled  measurements  to  have 
sub-pixel  shifts  with  respect  to  each  other  so  that  new  and 
independent  information  can  be  exploited  at  each  raw  measurement 
(even  though  pixelated)  to  digitally  synthesize  a  much  smaller  effective 
pixel  size  for  the  reconstructed  image. 

Herein,  we  introduce  a  fundamentally  new  pixel  super-resolution 
method  that  utilizes  wavelength  scanning  to  significantly  improve  the 
resolution  of  an  undersampled  or  pixelated  imaging  system,  without 
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the  use  of  any  lateral  shifts  or  displacements.  In  this  technique,  the 
specimen  is  sequentially  illuminated  at  a  few  wavelengths  that  are 
sampled  from  a  rather  narrow  spectral  range  of  10-30  nm.  Compared 
with  sub-pixel  displacement  or  lateral  shift-based  super-resolution 
techniques,  wavelength  scanning  introduces  a  uniform  resolution 
improvement  across  all  the  directions  on  the  sample  plane  and 
requires  significantly  fewer  raw  measurements  to  be  made.  Making 
fewer  measurements  without  sacrificing  performance  could  greatly 
benefit  high-speed  wide-held  imaging,  field-portable  microscopy  and 
telemedicine  applications,  which  are  all  sensitive  to  data  transmission 
and  storage. 

We  demonstrated  the  effectiveness  of  this  new  wavelength- 
scanning-based  pixel  super-resolution  approach  using  lens-free  holo¬ 
graphic  microscopy  (Figure  1)  to  improve  the  resolution  and  the 
effective  numerical  aperture  (NA)  of  a  unit  magnification  imaging 
system  by  a  factor  of  ~4  in  all  directions.  Using  12  different 
illumination  wavelengths  between  480  and  513  nm,  we  achieved  a 
half-pitch  resolution  of  ~  250  nm  and  an  effective  NA  of  ~  1.0  across  a 
large  FOV  of  >  20  mm2,  which  constitutes  a  space-bandwidth  product 
of  >  1  billion.  At  the  heart  of  these  results  is  an  iterative  pixel  super¬ 
resolution  algorithm  that  was  developed  to  obtain  high-resolution 
complex  (that  is,  phase  and  amplitude)  reconstructions  from  under¬ 
sampled  (that  is,  pixelated)  lens-free  digital  holograms  acquired  at 
different  wavelengths. 

In  the  previous  work,  wavelength  diversity  in  illumination  has  been 
mainly  utilized  for  two  general  purposes.  The  first  has  been  to  obtain 
color  or  spectral  information  of  a  sample13-25;  wavelength-dependent 
transmission,  absorption  or  scattering  features  of  the  specimen 
enhance  the  contrast  of  the  image  and  might  reveal  chemical  and/or 
physical  properties  of  biological  samples.  The  second  purpose  has  been 


to  recover  the  phase  of  the  optical  wavefront  in  digital  holography. 
Previous  studies26-29  have  demonstrated  a  phase  retrieval  technique  by 
tuning  the  wavelength  of  the  illumination  for  non-dispersive  objects. 
This  wavelength- diversity-based  phase  retrieval  approach  requires  the 
illumination  to  be  tuned  over  a  rather  large  spectral  range26-29  (that  is, 
>  60  nm)  and  assumes  that  the  specimen  maintains  similar  transmis¬ 
sion  properties  across  such  a  large  bandwidth.  Recently,  a  means  of 
using  wavelength  diversity  to  provide  a  modest  (for  example,  ~  16%) 
improvement  in  resolution  has  also  been  reported30.  However,  this 
method  also  assumes  that  the  transmission  properties  of  the  specimen 
remain  unchanged  across  an  even  larger  spectral  range  of  >  400  nm 
(that  is,  460-870  nm),  a  condition  that  would  not  be  satisfied  for 
realistic  samples,  including,  for  example,  pathology  slides  and  most 
biological  specimens. 

In  addition  to  significantly  improving  the  resolution  and  the  space- 
bandwidth  product  of  the  imaging  system,  our  wavelength- scanning 
pixel  super-resolution  approach  over  a  narrow  band  also  helps  in 
robust  phase  unwrapping  to  determine  the  optical  path  length 
differences  between  the  sample  and  surrounding  medium 
accurately31-33.  For  samples  in  which  the  optical  path  length  is  longer 
than  the  wavelength,  the  obtained  phase  map  will  be  wrapped.  In 
particular,  when  the  object  has  sharp  boundaries,  such  errors  may  be 
difficult  to  correct  using  state-of-the-art  phase  unwrapping  techniques 
based  on  a  single  wavelength  reconstruction34-37.  By  making  use  of  all 
the  illumination  wavelengths  in  our  super-resolution  approach,  we 
also  demonstrated  robust  phase  unwrapping  in  our  high- resolution 
microscopic  phase  images,  correctly  revealing  the  optical  path  length 
information  of  the  samples. 

In  addition  to  lens-free  holographic  imaging  techniques,  we 
experimentally  demonstrated  that  the  same  wavelength- scanning- 


Figure  1  Optical  setup  of  wavelength-scanning  pixel-super  resolution,  (a)  A  lens-free  holographic  on-chip  microscope  using  wavelength-scanning  pixel  super¬ 
resolution.  A  fiber-coupled  tunable  light  source  is  placed  above  the  object.  When  performing  pixel  super-resolution,  the  wavelength  is  scanned  within  a 
spectral  range  of  10-30  nm.  Multi-height  and  synthetic  aperture  imaging  configurations  are  also  integrated  into  this  setup  to  enable  phase  retrieval,  (b)  Lens- 
free  holograms  at  different  wavelengths  before  and  after  digital  sampling  at  the  image  sensor  plane. 
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based  super-resolution  framework  can  also  be  applied  to  improve  the 
resolution  of  lens-based  imaging  systems  with  the  introduction  of  a 
slight  defocus,  making  this  work  broadly  applicable  to  various 
coherent  or  partially  coherent  wide-held  imaging  modalities  that  are 
limited  by  pixelation  or  undersampling.  Therefore,  we  believe  that  this 
new  wavelength- scanning-based  super-resolution  technique  would 
largely  benefit  wide-held  and  high-throughput  microscopy  applica¬ 
tions  that  require  enhanced  space-bandwidth  products. 

MATERIALS  AND  METHODS 

Optical  setup 

As  depicted  in  Figure  1,  the  optical  setup  of  the  lens-free  microscope 
consists  of  three  major  components:  a  light  source,  an  image  sensor 
array,  and  a  specimen.  A  hber- coupled,  wavelength-tunable  light 
source  (WhiteLase-Micro,  model  VIS,  Fianium  Ltd,  Southampton, 
UK)  is  used  to  perform  the  wavelength  scanning.  During  the  imaging 
process,  the  central  wavelength  of  the  source  is  scanned  within  a 
spectral  range  of  10-30  nm  (for  example,  from  498  to  510  nm)  with  a 
step  size  of  ~  3  nm.  The  spectral  linewidth  of  illumination  at  each 
wavelength  is  ~  2  nm,  and  the  power  of  the  light  source  is  adjusted  to 
~  20  pW.  The  image  sensor  chip  is  a  color  CMOS  sensor  chip  with  a 
pixel  size  of  1.12  pm  manufactured  for  cellphone  camera  modules 
(IU081,  Sony  Corporation,  Tokyo,  Japan).  During  the  imaging  process, 
the  specimen  is  mounted  on  a  transparent  substrate  and  placed  100- 
500  pm  above  the  image  sensor  chip.  We  merged  our  wavelength- 
scanning-based  pixel  super-resolution  approach  with  both  multi¬ 
height11  and  synthetic  aperture  imaging12  configurations  to  obtain 
phase-retrieved,  high-resolution  reconstructions  of  the  specimen.  For 
synthetic-aperture-based  imaging12,  the  fiber  outlet  of  the  light  source 
is  mounted  on  a  rotational  arm  (PRM1Z8,  Thorlabs,  Newton,  NJ, 
USA),  and  the  image  sensor  is  placed  on  a  stage  that  can  rotate  over  a 
horizontal  plane.  Therefore,  the  incident  light  can  be  set  to  arbitrary 
illumination  angles,  which  is  required  for  the  synthetic  aperture 
approach.  For  multi-height-based  phase  retrieval9,11,  the  incremental 
height  change  between  the  image  sensor  and  the  specimen  is  enabled 
by  a  mechanical  positioning  stage  (MAX606,  Thorlabs).  The  image 
sensor  is  mounted  on  this  mechanical  stage,  whereas  the  specimen  is 
held  by  a  three-dimensional-printed  sample  holder.  After  completing 
image  capture  for  each  height,  the  stage  lowers  the  image  sensor  by  10- 
15  pm  on  average  before  the  image  capture  for  the  next  height  starts. 
During  the  imaging  process,  all  the  necessary  steps,  including  the 
wavelength  scanning  of  the  light  source,  multi-height  and  synthetic- 
aperture-related  scans  and  data  acquisition  using  the  image  sensor  chip 
are  automated  by  a  custom- written  Lab  VIEW  code  (Version  2011, 
National  Instruments,  Austin,  TX,  USA). 

Wavelength  calibration  and  dispersion  compensation 

Wavelength  calibration  of  our  light  source  is  achieved  using  an  optical 
spectrum  analyzer  (HR2000+,  Ocean  Optics,  Amersham,  UK).  The 
intensity-weighted  average  wavelength  of  each  measured  spectrum  is 
considered  as  our  illumination  wavelength.  To  achieve  optimal 
resolution,  the  refractive  index  of  the  glass  substrate  (100  pm, 
N-BK7,  Schott  AG,  Mainz,  Germany)  at  each  wavelength  is  also 
corrected  using  the  dispersion  formula  for  borosilicate  glass. 


(ethylenediaminetetraacetic  acid)  anticoagulated  human  blood  and 
stained  with  Wright’s  stain38. 

Mathematical  formalism  of  wavelength-scanning  pixel 
super-resolution 

We  assume  that  the  specimen  is  a  thin  object  mounted  on  a  plane 
parallel  to  the  image  sensor  chip  and  that  the  specimen  is  sequentially 
illuminated  by  multiple  wavelengths  {Xk}.  At  a  given  wavelength  Ak,  the 
object  wave  can  be  written  as  ok(x,  y)  —  l+sk(x,  y),  where  sk(x,  y) 
represents  the  scattered  object  wave,  immediately  at  the  exit  of  the 
object  plane  {z  =  0,  in  Figure  la).  The  two-dimensional  Fourier 
transform  of  ok(xy  y)  can  be  written  as  Ok(fx ,  fy)=5{f0O  fy)+Sk(fx ,  fy). 
At  the  image  sensor  plane  ( z  =  z0  in  Figure  la),  the  Fourier  transform 
of  the  intensity  distribution,  ik(x,  y),  can  be  written  as  (see 
Supplementary  Information  for  details): 


Ik  =  5  +  Tk-Sk+(Tk  -Sk)*  +SSk  (1) 

To  simplify  our  notation,  we  hide  the  expression  of  the  variables  for 
spatial  frequencies  (fx ,  fy),  and  the  superscript  c  -  ’  represents  (~fx, 
—fy).  On  the  right-hand  side  of  Equation  (1),  the  first  item,  <5, 
represents  the  background  intensity;  the  second  and  third  items  are 
conjugate  holographic  terms,  which  represent  the  interference  of  the 
scattered  object  wave  with  the  background  wave  at  the  sensor  plane. 
The  fourth  item  is  the  self-interference  term,  which  can  be  considered 
negligible  for  weakly  scattering  objects.  The  expression  for  Tk  can  be 
written  as  follows: 


TkifxJy)  =  H*k  (fXik,fytk)  ■  Hk  (/x  +fXtk,fy  +/M)  (2) 

Where  Hk(fx>  fy)  is  the  free  space  transfer  function,  and  the  frequency 
shifts  fX)k  and  fy>k  are  determined  by  the  illumination  wavelength  and 
the  incident  angle  (refer  to  the  Supplementary  Information  for 
details).  After  the  object  intensity  is  sampled  by  an  image  sensor 
array  with  a  pixel  pitch  of  Ax  and  Ay,  the  discrete  Fourier  transform 
of  the  sensor’s  output  can  be  expressed  as  follows39: 


/sampled,  k  —  ^  ^  /pix,  k  V-  '  Ax’L  Ay 


u,  v=0,  ±  1,.. 


(3) 


where  u  and  v  are  integers  and  fx  and  fy  are  discrete  spatial  frequency 
values.  Note  that  I^k(f„  fy)  =  Ik(fx,  fy>Pk(fx>  fy)>  where  Pk(f„  fy) 
represents  the  Fourier  transform  of  the  pixel  function,  that  is,  the 
two-dimensional  responsivity  distribution40  within  each  pixel:  pk(x,  y). 
Variables  u  and  v  represent  the  order  of  spatial  aliasing  due  to 
pixelation,  and  (u,  v)  =  (0,0)  corresponds  to  the  non-aliased  real  (that 
is,  target)  signal.  The  periodic  nature  of  the  discrete  Fourier  transform 
enables  us  to  extend  the  expression  of  /sampled,  k  to  a  broader  frequency 
space  by  upsampling  (Figure  2).  Based  on  these  definitions,  we  can 
express  the  undersampled  or  pixelated  lens-free  hologram  at  a  given 
wavelength  Xk  as  follows: 


Sample  preparation 

The  grating  lines  used  for  resolution  quantification  are  fabricated  on  a 
-100  pm  glass  slide  (N-BK7,  Schott  AG)  using  focused  ion  beam 
milling.  Unstained  Papanicolaou  (Pap)  smear  slides  are  prepared 
through  the  ThinPrep  method  (Hologic,  Inc.,  Marlborough,  MA, 
USA).  The  blood  smear  samples  are  prepared  using  EDTA 


/sampled,  k  —  /  J  |  &uv  T  TUVj  k  *  SUVj  k  ~\~  \  TUVj  k  ‘  SUVj  kJ  T  SSUVj  *  Puv,  k 

u,  v 

(4) 

The  non-aliased  target  signal  ok(x ,  y)  or  its  spatial  Fourier  spectrum 
can  be  obtained  under  («,  v)  =  (0,0),  that  is,  doo+^oo,  h  w^ich  can 
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i 


Forward 
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Amplitude  update  using 
low-resolution  measurements 
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High-resolution,  phase- 
retrieved  reconstructions 


Figure  2  Reconstruction  algorithm  for  wavelength-scanning  pixel  super-resolution,  integrated  with  multi-height  or  synthetic-aperture-based  lens-free  phase 
retrieval.  Refer  to  the  Materials  and  Methods  section  for  further  details.  The  fourth  column  shows  super-resolved  and  phase-retrieved  reconstruction  of  the 
specimen.  FFT,  fast  Fourier  transform;  IFFT,  inverse  fast  Fourier  transform. 


written  as  follows: 

(^00  T~  Soo,  k)  '  Poo,  k  =  Too,  k  '  -^sampled,  k  ~  ^00,  k  '  Tqq,  k  •  PoO,  k  '  >^00,  k 

—  (T00,  k  '  T« V,  k  ■  Pwv,  k  ■  Suv,  k  +  T00j  k  •  k  ’  PMVj  fc  *  SUVj 

M/0,  v^O  X 

-  E  Tkk-^-  E  ^00,  k  '  PUV,  k  '  SS„v,  k 

U  +  0,  V/0  M,  v=0,  ±1,... 

(5) 

On  the  left  side  of  Equation  (5),  we  retain  the  pixel  function  Poo,  h 
which  can  be  removed  later  in  the  last  step  of  the  image  reconstruc¬ 
tion,  using,  for  example,  spatial  deconvolution  with  a  Wiener  filter41,  as 
illustrated  in  ref.  40.  Equation  (5)  also  shows  that  to  obtain  the  non- 
aliased  object  at  (w,  v)  =  (0,0),  one  needs  to  eliminate  or  subtract  four 
terms  from  the  upsampled  and  back-propagated  holographic  term  (that 
is,  T*0  k  •  /sampled,  k)-  To  this  end,  the  first  item  to  eliminate, 
Too  k  '  Too  k  ■  p00,  k  ‘  Sqo*  k’  the  twin  image  noise,  a  characteristic 
artifact  of  in-line  holography.  The  second  term  in  Equation  (5),  which 
contains  SUVj  k  and  S~*  k  (m#  0,  v^0)  in  the  summation,  represents  the 
effects  of  spatial  aliasing  and  undersampling  for  both  the  real  and  twin 
image  terms.  The  third  item,  which  contains  buv  in  the  summation,  is 
the  periodic  background  artifact  generated  during  the  upsampling 
process,  and  the  last  item  is  the  self-interference  term  and  its 
upsampling  related  artifacts.  Starting  with  the  next  sub-section,  we  will 
discuss  a  two-stage  reconstruction  algorithm  for  eliminating  all  four  of 
these  items  listed  on  the  right  side  of  Equation  (5)  using  wavelength 
scanning  to  enable  super-resolved  reconstructions  of  complex  (that  is, 
phase  and  amplitude)  object  functions. 

Reconstruction  Stage  1:  generation  of  a  high-resolution  initial  guess 
of  the  specimen  using  wavelength  diversity 

As  depicted  in  Figure  2,  the  reconstruction  of  the  specimen  image 
involves  two  stages.  First,  it  should  be  noted  that  in  Equation  (5)  the 


functions  {Tq0  k  ’  Tuv ,  k  {u=^ 0,  v^0)}  have  complex  values  with  unit 
amplitude,  and  their  phases  are  highly  sensitive  to  changes  in 
wavelength  (see  the  Supplementary  Information  for  details).  There¬ 
fore,  when  the  illumination  wavelength  (Ak)  is  scanned  over  K 
different  wavelengths  that  are  uniformly  spread  across  a  narrow 
bandwidth,  the  set  of  functions  {Tq0  k  ’  Tuv ,  k  («^0  or  v^0)}  can  be 
considered  rotating  unit  vectors,  and  by  summing  all  these  rotating 
vectors  as  a  function  of  wavelength,  we  obtain  the  following: 


K 

Too,  k 

k=  1 


<<*:•  \puv,k 


(6) 


This  expression  indicates  that  by  summing  all  the  back  propagations  at 
different  wavelengths  (for  example,  over  a  narrow  spectral  range  of 
10-30  nm),  the  reconstructed  image,  that  is,  d0o  +  S0o,  k  or 
(doo  +  Soo,  k )  '  Poo,k>  can  be  significantly  enhanced,  whereas  the  spatial 
aliasing  and  undersampling-related  terms  with  Tq0  k  •  TUV)  k  will  be 
considerably  suppressed.  Therefore,  in  this  first  stage  of  our  recon¬ 
struction  process,  we  generate  a  high- resolution  initial  guess  of  the 
specimen  by  summing  all  the  upsampled  and  back-propagated  raw 
measurements,  that  is,  low-resolution  diffraction  patterns.  We  subtract 
the  artifact  items  {Suv  («^0,  v/0)}  before  the  back- propagation  step  to 
create  a  cleaner  image. 

It  should  be  noted  that  modifying  Equation  (6)  into  a  weighted 
average  at  each  spatial  frequency  point  could  achieve  better  suppres¬ 
sion  of  spatial  aliasing  and  undersampling- related  artifacts.  However, 
using  our  current  computation  platform,  which  is  based  on  a  central 
processing  unit,  the  search  for  optimal  weighting  factors  at  each 
frequency  point  will  significantly  increase  the  total  computation  time. 
Therefore,  in  this  proof-of-concept  implementation,  we  choose  a 
simpler  summation  approach  to  minimize  the  computation  time  for 
the  generation  of  the  initial  object  guess.  The  spatial  aliasing  and 
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undersampling-related  artifacts  of  this  initial  guess  will  be  further 
eliminated  and  cleaned  up  during  the  second  stage  of  our  algorithm, 
as  will  be  detailed  next. 

Reconstruction  Stage  2:  multi-wavelength-based  iterative  pixel 
super-resolution  and  phase  retrieval 

The  second  stage  of  our  numerical  reconstruction  involves  an  iterative 
algorithm,  which  contains  four  sub-steps  in  each  iteration: 

(1)  Knowing  each  raw  measurement’s  corresponding  wavelength 
and  incidence  angle,  we  apply  the  corresponding  plane  wave  illumina¬ 
tion  on  the  initial  guess  of  the  specimen  (from  Stage  1,  discussed 
above)  and  propagate  the  optical  field  from  the  object  plane  to  the 
image  sensor  plane  using  the  angular  spectrum  approach42. 

(2)  The  amplitude  of  the  high- resolution  field  on  the  image  sensor 
plane  is  updated  using  the  low-resolution  measurement  at  the 
corresponding  wavelength.  To  this  end,  the  intensity  of  the  high- 
resolution  field  is  convolved  with  the  image  sensor’s  pixel  function 
and  downsampled  to  the  same  grid  size  as  the  pixelated  raw 
measurement.  The  difference  between  the  raw  measurement  and  the 
downsampled  intensity  map  is  considered  a  low-resolution  ‘correc¬ 
tion’  map  for  each  illumination  wavelength.  A  high- resolution 
correction  map  can  then  be  generated  by  taking  the  Kronecker 
product  of  this  low-resolution  map  and  the  pixel  function.  To  perform 
a  smooth  update,  this  high-resolution  correction  map  is  added  to  the 
high- resolution  intensity  distribution  with  a  relaxation  parameter, 
typically  set  to  ~0.5  (see  the  Supplementary  Information  for  details). 
After  the  smoothed  update,  a  Wiener  deconvolution  filter  that 
incorporates  the  image  sensor’s  noise  level  is  applied  to  this  updated 
intensity  distribution.  The  square  root  of  this  filtered  high-resolution 
intensity  distribution  is  then  applied  to  the  amplitude  of  the  field  on 
the  sensor  plane  while  the  phase  map  is  kept  unaltered. 

(3)  This  updated  field  is  then  back-propagated  to  the  object  plane. 

(4)  The  back-propagated  field  is  used  to  update  the  transmission 
field  on  the  object  plane.  This  update  is  performed  in  the  frequency 
domain  (Figure  2)  within  a  circular  area  whose  center  is  determined 
by  the  corresponding  illumination  wavelength  and  angle.  The  radius  of 
this  circle  is  defined  by  the  boundary  within  which  all  the  spatial 
frequencies  experience  an  attenuation  of  less  than  3  dB  after  propaga¬ 
tion  in  the  spatial  domain.  This  update  on  the  object  plane  is  also 
smoothed  using  a  relaxation  factor  of  -0.5.  After  the  update,  the 
phase  of  the  field  on  the  object  plane  is  converted  to  an  optical  path 
length  map,  and  its  amplitude  is  directly  used  as  the  transmission  of 
the  object. 

The  four  steps  described  above  are  performed  for  every  raw  (that  is, 
undersampled)  measurement  captured  by  the  image  sensor  array.  It  is 
considered  one  iteration  cycle  when  each  one  of  the  raw  measure¬ 
ments  has  been  used  for  amplitude  update.  Typically  after  5-10 
iteration  cycles,  the  reconstruction  converges.  The  convergence  con¬ 
dition  for  the  iteration  is  defined  as  follows43: 

|  SSEf^-SSEZg'\<e  (7) 

where  SSEl*g  is  the  sum- squared  error  between  the  raw  measurement 
and  the  downsampled  intensity  map43,  c itr  is  the  iteration  number, 
and  s  is  a  convergence  constant  determined  by  the  noise  level  of  the 
raw  (that  is,  undersampled)  measurements. 

Phase  retrieval  using  multi-height  and  synthetic  aperture 
techniques 

Multi-height9,1 1,44-46  and  synthetic  aperture12  techniques  have  been 
proven  to  be  robust  phase  retrieval  methods  for  lens- free  on-chip 


imaging.  In  previously  reported  lens-free  reconstructions9,11,12,  pixel 
super-resolution  and  phase  retrieval  are  carried  out  sequentially:  at 
each  height  or  illumination  angle,  lateral  shift-based  pixel  super¬ 
resolution  is  first  performed  to  obtain  high-resolution  diffraction 
patterns  on  the  image  sensor  plane.  These  super-resolved  diffraction 
patterns  are  then  used  by  an  iterative  phase  retrieval  algorithm,  in 
which  wave  propagations  between  the  object  plane  and  the  image 
sensor  plane  are  executed  repeatedly9,11,12.  However,  in  wavelength- 
scanning-based  pixel  super-resolution,  raw  measurements  are  essen¬ 
tially  undersampled  versions  of  different  holograms.  Therefore,  we 
choose  to  use  the  same  iterative  algorithm  detailed  in  the  previous 
sub-section  (that  is,  Reconstruction  Stage  2)  to  realize  resolution 
enhancement  and  phase  retrieval  altogether.  More  specifically,  in  the 
multi-height  configuration,  the  specimen  is  illuminated  sequentially  at 
each  wavelength,  and  the  corresponding  lens-free  holograms  are 
captured  before  the  vertical  scanning  stage  moves  the  sample  or  the 
image  sensor  to  the  next  height.  Each  height  will  be  labeled  with  index 
/;  therefore,  all  the  measurements  {/sampled>  k\  and  the  corresponding 
transfer  functions  {//*.}  and  {TUVy  *.}  that  are  used  in  the  previous 
derivations  can  be  relabeled  as  {/sampled*  £/}*  {Z/jy }  and 
{Tuv,  ]y}>  respectively.  During  the  numerical  reconstruction  process, 
all  the  raw  holograms  are  upsampled,  back-propagated,  and  then 
summed  together  to  generate  the  high- resolution  initial  guess  at  a 
given  height.  In  Stage  2  of  our  reconstruction  algorithm,  the 
aforementioned  four-step  process  is  applied  to  each  raw  measurement. 
The  same  set  of  operations  and  processing  also  apply  to  the  synthetic 
aperture  technique12,  except  that  index  l  now  refers  to  each  illumina¬ 
tion  angle  instead  of  sample  height. 

In  general,  for  pathology  slides  such  as  blood  smears  and  Pap 
smears,  the  optical  path  length  difference  between  the  specimen  (that 
is,  biological  tissue)  and  the  medium  (that  is,  air  or  the  sealing  glue)  is 
rather  small.  Under  these  circumstances,  phase  unwrapping  is  not  a 
concern;  therefore,  in  the  phase  recovery  process,  we  can  use  a 
scrambled  order  of  {/sampled,  kl)  in  each  iteration  cycle.  However,  when 
working  with  samples  with  larger  optical  path  length  differences,  such 
as  grating  lines  carved  into  a  glass  substrate,  one  extra  step,  that  is, 
phase  unwrapping,  must  be  added  after  the  reconstruction,  and  the 
order  of  iterations  must  be  modified  accordingly,  which  will  be 
detailed  in  the  next  sub-section. 

Multi-wavelength  phase  unwrapping 

A  robust  phase  unwrapping  algorithm  requires  high-resolution  and 
phase-retrieved  reconstructions  at  multiple  wavelengths;  therefore,  we 
divide  the  raw  measurements  into  subsets,  in  which  the  wavelengths 
are  identical  or  very  similar  (for  example,  A2  <  5  nm),  and  perform 
the  four-step  reconstruction  process  previously  discussed  (as  part  of 
the  Reconstruction  Stage  2)  on  each  subset  separately.  For  example, 
reconstruction  number  1  uses  subset  {/sampled*  kl  I  k  =  1*  U-.-T},  No. 

2  uses  {/Sampled>  kl  \  k  =  2,  l=lt...L]  and  so  on.  When  the  iterations  for 
all  these  subsets  are  completed,  we  obtain  high-resolution  (that  is, 
super-resolved)  phase-retrieved  reconstructions  at  multiple  wave¬ 
lengths,  that  is,  {Ck},  whose  phase  maps  {<fe,  wrapped!  need  unwrapping. 
Using  these  wrapped  phase  maps  {^wrapped!  at  multiple  wavelengths, 
we  perform  phase  unwrapping  to  accurately  reveal  the  optical  path 
length  differences  between  the  specimen  and  the  surrounding 
medium.  Assuming  that  the  optical  path  length  difference  is  AL(x, 
y),  the  phase  distribution  at  the  object  plane  at  each  wavelength  can  be 
written  as  (p^x ,  y)  =  2;r-AL(x,  y)/2^.  The  wrapped  phase  can  thus  be 
expressed  as  ^wrapped  (*»  y)  =  <Pk(*>  y)  ±  2A&r,  where  -n<  ^wrapped  < 
n  and  N  is  an  integer.  These  resulting  wrapped  phase  maps  {(p^ 
wrapped!  that  are  generated  through  super- resolved  and  phase-retrieved 
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Reconstruction  from  single 
measurement  (z=  320  |um) 


Lateral  shift-based  pixel  super-resolution  (z=  320  pm) 


Wavelength  scanning 
pixel  super-resolution  (z=320  pm) 


Figure  3  Resolution  improvement  introduced  by  wavelength-scanning  pixel  super-resolution,  (a)  Reconstruction  from  a  single  raw  measurement  captured  by 
an  image  sensor  chip  with  a  pixel  pitch  of  1.12  pm.  (b,  c)  Lens-free  reconstructions  using  lateral  shift-based  pixel  super-resolution,  b  and  c  are  reconstructed 
using  super-resolved  holograms  synthesized  from  five  and  nine  sub-pixel-shifted  raw  measurements,  respectively.  The  corresponding  sub-pixel  shifts  of  the 
raw  measurements  are  marked  within  the  hologram  shift  tables,  (d)  Reconstruction  using  wavelength-scanning  pixel  super-resolution;  five  wavelengths  are 
used  within  a  scanning  range  of  498-510  nm  and  a  scan  step  size  of  3  nm. 


Lateral  shift-based  pixel  super-resolution 


5  shifted  images  at  each  height 
Total  5  heights 


9  shifted  images  at  each  height 
Total  5  heights 


Wavelength  scanning 
pixel  super-resolution 

q  5  wavelengths  used  at  each  height 
Total  5  heights 


Height:  330  pm:10  pm:370  pm 
X  =  510  nm 


Height:  330  pm:  10  pm:370  pm 
A,  =  510  nm 


Height:  330  pm:10  pm:370  pm 
X  =  498  nm:  3  nm:  510  nm 


Figure  4  Lens-free  imaging  using  multi-height  phase  retrieval  and  wavelength-scanning  pixel  super-resolution.  Five  heights  are  used  in  each  case  shown  in 
a-c.  (a,  b)  are  lens-free  reconstructions  using  lateral  shift-based  pixel  super-resolution  and  multi-height  phase  retrieval,  a  uses  five  and  b  uses  nine 
sub-pixel-shifted  raw  measurements  at  each  height,  (c)  Lens-free  reconstruction  using  wavelength-scanning  pixel  super-resolution  and  multi-height  phase 
retrieval;  five  wavelengths  are  used  with  a  scanning  range  of  498-510  nm  and  a  scan  step  size  of  3  nm. 


reconstructions  at  multiple  wavelengths  are  then  fed  into  an  optimiza¬ 
tion  algorithm47  that  finds  the  optimum  path  length  ALopt(x,y)  at  each 
spatial  point  ( x,y )  by  minimizing  a  cost  function  defined  as  follows: 

K 

|  (*,  y)  _  ‘  al°p1  7)  /h  | 2  ( g ) 

fc=i 

To  avoid  convergence  to  a  local  minimum  and  reduce  the  computa¬ 
tion  cost/time,  we  define  a  search  range  of  [  AL0  -  min{A^}/2,  AL0+min 
{2jJ/2],  where  AL0  is  the  initial  guess  of  the  optical  path  length: 


AL{ 


1  K 

>(*> A  ^271-iK  y)  ~  >01  ■ 


h-1  ^ k 


(9) 


where  the  total  number  of  wavelengths  (K)  is  typically  5-10.  Within 
this  search  interval,  we  scan  the  values  to  find  the  optical  path  length 
ALopt(x,  y)  that  minimizes  the  cost  function,  resulting  in  an 
unwrapped  object  phase  image. 


Computation  platform  used  for  super-resolved  image 
reconstructions 

Our  reconstructions  are  performed  using  MATLAB  (Version  R2012a, 
Math  Works,  Natick,  MA,  USA)  on  a  desktop  computer  equipped  with 
a  3.60-GHz  central  processing  unit  (Intel  Xeon  E5-1620)  and  16  GB  of 
random-access  memory.  For  a  1  X  1  mm2  sub-region  with  an  upsam¬ 
pling  factor  of  seven,  one  iteration  of  our  wavelength- scanning  super¬ 
resolution  routine  takes  ~  1.2  s.  For  example,  one  cycle  of  our 
algorithm,  which  undergoes  all  the  undersampled  measurements 
(for  example,  seven  wavelengths  for  each  angle/height,  and  22 
angles/heights  in  total),  takes  ~  3  min.  In  our  proof-of-concept 
implementation,  the  iterations  did  not  use  either  GPU  (graphics 
processing  unit)  or  parallel  computing,  which  could  significantly 
improve  our  overall  computation  time10.  The  total  image  reconstruc¬ 
tion  time  could  be  further  improved  by  implementing  the  algorithm 
in  the  C  language  rather  than  MATFAB. 

RESULTS  AND  DISCUSSION 

The  physical  basis  for  wavelength- scanning  pixel  super-resolution  is 
the  strong  wavelength  dependence  of  the  undersampled  interference 
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Lens-free  synthetic  aperture  reconstruction  Conventional  microscope 


Lateral  shift-based  Wavelength  scanning  60x  objective 

pixel  super-resolution  (z=  102  |um)  pixel  super-resolution  (z=  102  pm)  water  immersion,  NA  =  1.0 


Angle:  -40°:10°:40°,  1-axis  Angle: -40°:  10°:40°,  1-axis 

^  =  510  nm  X  =  480  nm:  3  nm  :513  nm 


Figure  5  (a)  Lens-free  imaging  using  synthetic  aperture  and  lateral  shift-based  pixel  super-resolution.  At  each  illumination  angle,  6x6  =  36  sub-pixel  lateral 
shifts,  which  are  spread  evenly  over  the  pixel  area,  are  used  for  super-resolution,  (b)  Lens-free  imaging  using  synthetic  aperture  and  wavelength-scanning 
pixel  super-resolution.  Twelve  wavelengths  are  used  within  a  scanning  range  of  480-513  nm  and  a  scan  step  of  3  nm.  The  illumination  angle  scanning 
directions,  ranges  and  scanning  steps  are  the  same  in  both  a  and  b.  (c)  Image  of  the  specimen  obtained  using  a  conventional  lens-based  microscope  with  a 
60 x  water-immersion  objective  lens  (NA=  1.0). 


patterns  in  coherent  or  partially  coherent  diffraction  imaging  systems 
such  as  lens-free,  holographic  microscopy  (Figure  1)  or  defocused 
lens-based  imaging  systems.  When  illuminated  at  slightly  different 
wavelengths,  the  high-frequency  interference  fringes  caused  by  object 
scattering  will  change,  causing  the  undersampled  output  of  the  image 
sensor  chip  to  change  as  well  (Figure  lb).  Our  derivations  (Materials 
and  Methods  section)  show  that  in  the  spatial  frequency  domain,  the 
aliasing  signal  caused  by  pixel-induced  undersampling  is  modulated  by 
a  complex  transfer  function  whose  phase  is  rather  sensitive  to  even 
small  wavelength  changes,  which  makes  it  possible  to  use  wavelength 
diversity  within  a  narrow  spectral  range  (that  is,  10-30  nm)  to  cancel 
out  the  spatial  aliasing  term  and  enhance  the  resolution  of  the 
reconstructions  beyond  the  pixel  pitch. 

This  spatial  resolution  improvement  brought  by  our  wavelength¬ 
scanning  pixel  super- resolution  technique  for  different  lens-free 
imaging  configurations  is  clearly  demonstrated  in  Figures  3-5.  With¬ 
out  pixel  super-resolution,  lens-free  microscopy  with  a  unit  magnifi¬ 
cation  on-chip  imaging  geometry  (in  which  the  sample  FOV  equals 
the  active  area  of  the  sensor  array)  can  achieve  a  half-pitch  resolution 
close  to  the  image  sensor’s  pixel  pitch  (that  is,  ~  1  pm  in  Figure  3a, 
using  a  CMOS  sensor  that  has  a  1.12-pm  pixel  pitch).  For  the  same 
configuration  depicted  in  Figure  3,  utilizing  the  wavelength  diversity  in 
illumination  boosts  the  half-pitch  resolution  to  ~0.6  pm  by  using  five 
different  wavelengths  between  498  and  510  nm  (Figure  3c  and  3d). 
When  integrated  with  the  multi-height  phase  retrieval  technique9,11, 
the  resolution  can  be  further  improved  to  -  0.5  pm,  corresponding  to 
an  effective  NA  of  -0.5  (Figure  4).  Using  the  synthetic  aperture 
technique12,  however,  can  provide  not  only  twin  image  elimination 
but  also  a  significant  increase  in  spatial  bandwidth  of  the  reconstructed 
images;  these  benefits  enable  us  to  take  full  advantage  of  the 
wavelength-scanning-based  pixel  super-resolution  technique  and 
achieve  a  half-pitch  resolution  of  -  250  nm  with  an  effective  NA  of 
-1.0  under  a  unit  magnification  geometry,  where  the  FOV  is 
>20  mm2  (Figure  5).  These  results  clearly  demonstrate  the  pixel 
super-resolution  capabilities  of  our  wavelength- scanning  approach. 

In  addition  to  delivering  a  competitive  resolution  and  NA,  the 
wavelength-scanning-based  super-resolution  approach  also  offers 
better  data  efficiency  compared  with  lateral  shift-based  pixel  super¬ 


resolution  techniques,  that  is,  fewer  raw  measurements  are  needed  for 
the  same  resolution  improvement.  In  lateral  shift-based  pixel  super¬ 
resolution,  the  sub-pixel  shifts  between  the  raw  measurements  are 
obtained  by  moving  the  light  source,  image  sensor  and/or  the 
specimen  with  respect  to  each  other7,8,11,48,  and  the  resolution 
improvement  is  direction  dependent.  Therefore,  sub-pixel  shifts  that 
spread  uniformly  within  a  two-dimensional  pixel  area  are  preferred  in 
lateral  shift-based  pixel  super-resolution  techniques  to  achieve  optimal 
resolution  enhancement.  As  a  result,  the  number  of  raw  measure¬ 
ments  generally  increases  as  a  quadratic  function  of  the  pixel 
super-resolution  factor.  However,  in  the  wavelength-scanning  super¬ 
resolution  approach,  the  resolution  improvement  caused  by  wave¬ 
length  diversity  is  uniform  in  all  lateral  directions  across  the  sensor 
array,  which  enables  us  to  achieve  competitive  resolution  with  much 
fewer  raw  measurements  compared  with  lateral  shift-based  super¬ 
resolution.  To  exemplify  this  important  advantage  of  our  approach,  in 
a  normal-illumination  configuration,  compared  with  the  lateral  shift 
technique,  which  needs  nine  measurements  to  achieve  a  half-pitch 
resolution  of  -  0.6  pm  (Figures  3b  and  3c),  the  wavelength- scanning 
technique  requires  only  five  raw  measurements  (Figure  3d)  to  reach 
the  same  imaging  performance.  Similarly,  when  combined  with  multi¬ 
height  phase  retrieval9,11,  wavelength- scanning  super-resolution 
requires  25  raw  measurements  in  total  (five  wavelengths  at  each  of 
five  heights),  which  is  significantly  smaller  than  the  number  required 
using  lateral  shifts  (that  is,  45  raw  measurements  as  shown  in 
Figure  4).  When  integrated  with  synthetic-aperture-based  lens-free 
imaging12,  an  even  higher  spatial  resolution  can  be  achieved  using 
wavelength  diversity,  and  the  advantages  of  wavelength  scanning  over 
lateral  shifts  become  more  significant.  As  shown  in  Figure  5,  to  achieve 
a  half-pitch  resolution  of  ~  250  nm  in  all  directions,  the  lateral  shift- 
based  super-resolution  technique  requires  a  total  of  612  images  (6x6 
raw  images  at  each  one  of  the  17  illumination  angles),  whereas  the 
presented  wavelength-scanning  technique  only  needs  204  images  (12 
wavelengths  at  each  one  of  the  17  angles),  corresponding  to  a  3 -fold 
reduction  in  the  number  of  images.  Taking  into  account  the  wait  time 
for  the  mechanical  components  in  our  bench-top  system,  the  lateral 
shift-based  technique  at  multiple  illumination  angles  takes  -21.5  min 
in  total,  whereas  our  wavelength- scanning  technique  takes  only 
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Figure  6  Lens-free  imaging  of  a  Papanicolaou  (Pap)  smear  using  wavelength-scanning  pixel  super  resolution  and  multi-height  phase  retrieval, 
(a)  Super-resolved  lens-free  phase  image  of  the  Pap  smear.  Wavelength-scanning  range:  510-531  nm,  scanning  step:  3  nm.  (b,  e)  Regions  of  interest  (ROI) 
No.  1  and  2,  respectively,  (c,  f)  Digital  phase  contrast  images  generated  from  lens-free  reconstructions  (refer  to  the  Materials  and  Methods  section  for 
details),  (d,  g)  Conventional  lens-based  phase  contrast  images  obtained  using  a  40 x  objective  lens  (NA  =  0.6). 
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Figure  7  Lens-free  imaging  of  a  blood  smear  using  wavelength-scanning  pixel  super-resolution  and  synthetic  aperture  phase  retrieval.  Wavelength-scanning 
range:  520-541  nm,  scanning  step:  3  nm.  Angle  scanning  (two  axes)  range:  -35°  to  35°  with  5°  steps,  (a)  Lens-free  raw  hologram  captured  using  a  CMOS 
image  sensor  with  a  pixel  size  of  1.12  pm  and  a  FOV  of  -20.5  mm2,  (b)  Lens-free  reconstruction  of  a  sub-region,  marked  with  a  yellow  square  within  the  full 
FOV.  (c)  Conventional  lens-based  microscope  image  using  a  20 x  objective  lens  (NA  =  0.45). 


~  4.8  min,  showing  a  significant  improvement  in  the  imaging  time. 
This  important  advantage  in  terms  of  the  reduced  number  of 
measurements  can  also  be  translated  into  smaller  data  storage  space, 
which  is  critical  for  increasing  the  speed  and  utility  of  high- resolution 
wide-held  imaging  techniques. 

We  should  re-emphasize  that  wavelength- scanning  super-resolution 
requires  only  a  few  wavelengths  taken  from  a  narrow  spectral  range 
(for  example,  10-30  nm).  With  this  new  super-resolution  approach, 
we  can  obtain  high- resolution  amplitude  reconstructions  of  not  only 
colorless  but  also  colored  (that  is,  stained/dyed)  samples  without 
further  modifications  in  our  reconstruction  algorithm.  We  demon¬ 
strated  this  capability  by  imaging  various  biological  samples,  including 
unstained  Pap  smears  (Figure  6)  as  well  as  stained  blood  smears 
(Figure  7).  Because  our  lens-free  reconstructions  provide  both 
amplitude  and  phase  channels,  we  can  also  visualize  the  reconstructed 
images  using  different  methods  to  create,  for  example,  a  digital  phase 
contrast  image12,49  as  illustrated  in  Figure  6. 


In  addition  to  significantly  improving  resolution  and  mitigating 
undersampling,  wavelength  diversity  also  enables  us  to  perform  robust 
phase  unwrapping  and  reveal  the  optical  path  length  differences 
between  a  given  specimen  and  surrounding  medium.  The  retrieved 
phase  reconstruction  from  a  single  wavelength  is  constrained  to  its 
principle  value  (- n ,  n\,  and  therefore  large  optical  path  length 
differences  can  cause  polarity  errors  that  may  not  be  corrected  even 
by  using  state-of-the-art  phase  unwrapping  algorithms34-37  (see,  for 
example,  Figure  8a  and  8b).  Such  polarity  errors  in  phase  reconstruc¬ 
tions  can  also  be  mitigated  by  detecting  the  phase  differences  between 
the  reconstructions  at  two  different  wavelengths33,50.  However,  this 
two -wavelength  phase  unwrapping  approach  still  faces  the  challenge  of 
ambiguity  or  uniqueness50.  In  addition  to  achieving  pixel  super¬ 
resolution,  we  further  utilized  wavelength  scanning  to  significantly 
improve  the  robustness  of  phase  unwrapping  by  incorporating  all  the 
wavelengths  of  illumination  that  are  used  in  pixel  super-resolution  to 
unwrap  the  reconstructed  phase  (Materials  and  Methods  section).  The 
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Figure  8  Phase  unwrapping  using  multiple  wavelengths.  The  sample  consists  of  four  grating  lines  carved  into  a  glass  substrate  using  focused  ion  beam 
milling,  (a)  Lens-free  phase  image  obtained  using  a  single  wavelength;  phase  map  is  wrapped,  (b)  Unwrapped  lens-free  phase  image  obtained  from  a  single 
wavelength  reconstruction  using  a  two-dimensional  phase  unwrapping  technique  based  on  minimum  network  flow36,  (c)  Unwrapped  lens-free  phase  image 
obtained  using  reconstructions  from  multiple  wavelengths.  Eleven  wavelengths  are  used  within  a  scanning  range  of  501-531  nm  and  a  scan  step  size  of 
3  nm.  Refer  to  the  Materials  and  Methods  section  for  further  details  on  multi-wavelength  phase  unwrapping,  (d-f)  Depth  profiles  of  a-c,  respectively.  The 
blue  dashed  line  in  f  is  the  depth  profile  of  the  sample  gratings  measured  using  an  atomic  force  microscope  (AFM). 
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Figure  9  Implementation  of  wavelength-scanning  pixel  super-resolution  using  a  lens-based  microscope,  (a)  Schematic  of  the  optical  setup.  In  this  proof-of- 
concept  experiment,  a  lOx  objective  lens  (NA  =  0.3)  is  used.  The  sample  plane  is  scanned  from  160  to  300  pm,  defocused  from  the  focal  plane  of  the 
objective  lens  in  20-pm  increments.  The  illumination  wavelength  is  tuned  between  500  and  521  nm  with  a  step  size  of  3  nm.  (b)  Full  FOV  of  a  CMOS  image 
sensor  (pixel  size:  3.75  pm)  using  the  10 x  objective  lens  with  a  0.38 x  camera  adaptor,  (c)  Lens-based  image  of  a  resolution  test  chart  without  wavelength¬ 
scanning  pixel  super-resolution.  The  half-pitch  of  the  smallest  resolved  feature  is  1.95  pm.  (d)  Lens-based  image  of  the  same  resolution  test  chart  with 
wavelength-scanning  pixel  super-resolution.  The  half-pitch  of  the  smallest  resolved  feature  is  0.78  pm. 


phase  unwrapping  results  shown  in  Figure  8c,  8f  clearly  illustrate  that 
through  optimization  we  can  entirely  rule  out  incorrect  optical  path 
length  differences  within  our  reconstructed  images  and  achieve  robust 
phase  unwrapping  in  a  super-resolved  image. 

The  wavelength-scanning  pixel  super-resolution  approach,  together 
with  phase  retrieval  methods,  including  multi-height9,11,  synthetic 


aperture12  and  object-support-based  techniques51,52,  could  constitute 
high-resolution  imaging  systems  with  greatly  improved  imaging  speed. 
For  a  bench-top  system,  high-speed  wavelength  scanning  can  be  realized 
using  a  fast  tunable  source  (employing,  for  example,  an  acousto-optic 
tunable  filter  with  a  supercontinuum  source)  synchronized  with  the 
image  sensor  chip.  Compared  with  lateral  shift-based  super-resolution 
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setups,  such  a  combination  avoids  motion  blur  and  could  increase  the 
data  acquisition  speed  to  the  maximum  frame  rate  of  the  image  sensor. 
Furthermore,  the  lateral  shifts  generated  by  the  source-shifting 
approach7-9,12  are  determined  by  both  the  sample-to-sensor  and 
sample-to-aperture  distances,  which  can  make  it  challenging  to  generate 
optimized  lateral  shifts  for  samples  at  different  vertical  heights.  The 
wavelength-scanning  approach,  however,  is  performed  with  evenly 
distributed  wavelengths  regardless  of  sample  height.  Therefore,  we 
believe  that  wavelength-scanning  pixel  super-resolution  is  more  favor¬ 
able  that  lateral  shifting  techniques  for  building  high-resolution  wide- 
held  microscopes  with  high  imaging  speeds.  In  addition,  the  better  data 
efficiency  of  our  wavelength-scanning  super-resolution  approach  can 
reduce  the  cost  of  data  storage  and  transmission,  benefiting  telemedicine 
implementations  and  server-based  remote  reconstructions. 

In  addition  to  delivering  competitive  results  on  a  bench-top  system, 
the  presented  wavelength- scanning  pixel  super-resolution  approach 
also  holds  great  potential  for  field-portable  microscopy  applications. 
Compared  with  lateral  shift-based  pixel  super-resolution,  the 
wavelength- scanning  approach  does  not  require  any  mechanical 
motion  or  fiber  bundle8,52,  which  could  make  the  mobile  imaging 
platform  more  robust.  Because  the  wavelength  scanning  range  is 
narrow  (that  is,  10-30  nm),  the  combination  of  a  few  light-emitting 
diodes,  each  with  a  standard  spectral  bandwidth  of  15-30  nm,  and  a 
variable  optical  thin-film  filter  to  narrow  the  light-emitting  diode 
spectra  will  be  sufficient  to  implement  wavelength-scanning  super¬ 
resolution  in  a  field-portable  and  cost-effective  design. 

Finally,  we  should  emphasize  that  the  presented  wavelength¬ 
scanning  pixel  super-resolution  approach,  in  addition  to  lens-free  or 
holographic  diffraction-based  imaging  systems,  can  also  be  applied  to 
lens-based  point-to-point  imaging  modalities.  By  introducing  a  simple 
defocus  into  a  lens-based  imaging  system  (by,  for  example,  a  relative 
axial  shift  of  the  sensor  array,  object  and/or  the  objective  lens),  the 
entire  wavelength  diversity  framework  described  in  this  manuscript 
would  be  able  to  achieve  pixel  super-resolution.  For  proof-of-concept, 
we  experimentally  demonstrated  our  wavelength- scanning  pixel  super¬ 
resolution  approach  using  a  conventional  lens-based  microscope  with 
a  10  x  objective  lens  (NA  =  0.3)  and  a  CMOS  sensor  chip  with  a  pixel 
size  of  3.75  pm  (Figure  9).  To  significantly  expand  the  FOV  of  the 
microscope  from  -0.17  to  1.20  mm2,  we  used  a  0.38  X  camera 
adaptor,  which  created  an  effective  pixel  size  of  -  0.99  pm,  limiting  the 
resolution  of  the  microscope  because  of  pixelation  (Figure  9c).  To 
considerably  improve  the  resolution,  back  to  the  resolving  capability  of 
the  objective  lens  (dictated  by  its  NA),  we  defocused  the  target  object 
by  small  increments  of  -20  pm  between  160  and  300  pm.  We  also 
scanned  the  illumination  wavelength  between  500  and  521  nm  with  a 
step  size  of  3  nm.  Using  these  defocused  microscope  images  in  our 
wavelength- scanning  super- resolution  algorithm,  we  significantly 
improved  the  resolution  of  the  reconstructed  image  (Figure  9d), 
matching  the  resolution  determined  by  the  NA  of  the  objective  lens 
while  also  expanding  the  image  FOV  by  -  sevenfold  (that  is,  1.20  vs 
0.17  mm2).  These  results  confirm  that  our  wavelength- scanning  pixel 
super-resolution  technique  is  broadly  applicable  to  various  lens-based 
and  lens-free  coherent  and  partially  coherent  wide-held  imaging 
modalities. 

CONCLUSIONS 

We  report  a  new  wavelength- scanning-based  pixel  super-resolution 
technique  that  generates  high-resolution  reconstructions  from  under¬ 
sampled  raw  measurements  captured  at  multiple  wavelengths  within  a 
narrow  spectral  range  (that  is,  10-30  nm).  Compared  with  lateral 
shift-based  super- resolution,  this  wavelength- scanning  method  avoids 


the  need  for  shifting  mechanical  components  and,  more  importantly, 
provides  uniform  resolution  improvement  along  all  the  directions 
across  the  image  sensor  or  sample  plane.  This  framework  enabled  us 
to  improve  the  resolution  and  effective  NA  of  a  wide-held  lens-free 
microscope  by  a  factor  of  -4,  achieving  a  half-pitch  resolution  of 
250  nm  with  an  NA  of  -  1.0  using  signihcantly  fewer  measurements 
compared  with  those  required  by  lateral  shift-based  super-resolution 
methods.  Because  this  wavelength- scanning  super-resolution  techni¬ 
que  utilizes  a  narrow  spectral  range,  it  allows  for  super-resolved 
imaging  of  both  colorless  (for  example,  unstained)  and  stained/dyed 
biological  samples.  Reconstructions  at  multiple  wavelengths  also 
enable  robust  phase  unwrapping  to  reveal  the  optical  path  length 
differences  between  specimens  and  surrounding  media.  This  new 
wavelength-scanning  super- resolution  approach  would  broadly  beneht 
various  lens-free  as  well  as  lens-based  wide-held  and  high-throughput 
microscopy  applications  that  require  large  space-bandwidth  products. 
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